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U.  S.  Army  Research  Office  Contract  DAAL03-91-G-0032 
1  February  1991  -  30  June  1994 


Principal  Investigator: 

Stanley  C.  Eisenstat,  Department  of  Computer  Science,  Yale  University 


Statement  of  the  Problem  Studied: 

The  project  entailed  investigating  several  problems  in  the  parallel  solution  of  sparse 
systems  of  linear  equations  and  eigenproblems,  including: 

•  algorithms  for  factoring  sparse  matrices; 

•  techniques  for  the  solution  of  sparse  triangular  systems; 

•  iterative  methods  for  sparse  systems,  focusing  mainly  on  preconditioning  techniques  for 
conjugate-direction  methods; 

•  the  solution  of  the  symmetric  tridiagonal  eigenproblem. 

While  this  may  seem  to  be  an  eclectic  group  of  topics,  there  are,  in  fact,  close  relationships 
among  them.  As  one  example,  a  common  technique  for  preconditioning  iterative  methods 
depends  crucially  on  efficient  solution  of  triangular  systems.  As  another,  it  should  be  possible 
to  construct  an  effective  Lanczos-type  algorithm  for  sparse,  symmetric  eigenproblems  by 
combining  the  techniques  required  for  conjugate-direction  iterations  for  linear  systems  with 
those  required  for  the  solution  of  symmetric  tridiagonal  eigenproblems. 


Summary  of  the  Most  Important  Results: 

The  main  focus  has  been  on  algorithms  for  the  symmetric  tridiagonal  eigenvalue  problexn, 
including  the  bidiagonal  singular  value  problem  and  the  problem  of  updating  and  downdating 
the  singular  value  decomposition  (SVD): 

•  We  have  devised  a  stable  and  efficient  algorithm  for  finding  the  spectral  decomposi¬ 
tion  of  a  matrix  that  is  the  sum  of  a  diagonal  matrix  and  a  rank-one  matrix.  This 
is  the  key  subproblem  in  divide-and-conquer  algorithms  for  the  symmetric  tridiagonal 
eigenproblem  (including  Cuppen’s  method  [Cuppen,  Dongarra+ Sorensen]),  and  in  com¬ 
puting  the  spectral  decomposition  of  a  symmetric  rank-one  modification  of  a  symmetric 
matrix  whose  eigendecomposition  is  known.  All  previous  approaches  required  the  use 
of  extended  precision  arithmetic  in  order  to  maintain  orthogonality  of  the  eigenvectors 
[Kahan,  Sorensen+Tang],  The  new  algorithm  is  based  on  a  novel  and  stable  method  for 
computing  the  eigenvectors  (and  thus  does  not  require  extended  precision),  yet  it  is  as 
efficient  as  previous  approaches  and  just  as  parallel. 

•  We  have  devised  a  new  arrowhead  divide-and-conquer  method  for  the  symmetric  tridi¬ 
agonal  eigenproblem;  a  new  divide-and-conquer  method  for  finding  the  SVD  of  a  bidi¬ 
agonal  matrix;  and  new  methods  for  updating  and  downdating  the  SVD  of  a  dense 
matrix  when  a  row  or  column  is  added.  The  approach  above  is  used  to  stabilize  each 
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of  these  algorithms.  Moreover,  we  have  shown  how  to  update  the  orthogonal  factor(s) 
more  efficiently  than  previous  algorithms  do  by  taking  full  advantage  of  the  nonzero 
structure  without  reducing  parallelism. 

•  We  have  shown  how  to  use  the  fast  multipole  method  of  Greengard  and  Rokhlin  to 
accelerate  these  procedures.  For  example,  the  straight-forward  implementation  of  the 
arrowhead  divide-and-conquer  algorithm  computes  all  the  eigenvalues  of  an  N  x  N 
matrix  in  0(N 2)  time,  and  both  the  eigenvalues  and  eigenvectors  in  0(N 3)  time.  The 
accelerated  method  computes  all  the  eigenvalues  in  0(N  \og2  N)  time,  and  both  the 
eigenvalues  and  eigenvectors  in  0(N 2)  time. 

Another  focus  has  been  the  problem  of  computing  the  rank-revealing  QR  factorization. 
A  new,  strong  rank-revealing  QR  decomposition  of  a  matrix  has  been  defined.  Unlike  the 
original  rank- revealing  QR  decomposition  of  [Chan],  the  new  decomposition  leads  to  stable 
algorithms  for  finding  a  basis  for  an  approximate  right  null  space  of  a  matrix  or  for  separating 
the  linearly  independent  columns  of  a  matrix  from  the  linearly  dependent  ones.  The  existence 
of  such  decompositions  has  been  proved,  and  an  algorithm  has  been  devised  that  computes 
them  in  nearly  the  same  time  as  the  ordinary  QR  decomposition. 

A  third  focus  has  been  the  discovery  of  a  new  technique  for  deriving  bounds  on  the 
relative  change  in  the  singular  values  of  a  real  matrix  (or  the  eigenvalues  of  a  real  symmetric 
matrix)  due  to  a  certain  class  of  perturbations,  as  well  as  bounds  on  the  angles  between  the 
unperturbed  and  perturbed  singular  vectors  (or  eigenvectors).  This  class  of  perturbations 
include  component-wise  relative  perturbations  of  the  entries  in  a  bidiagonal  or  biacyclic  ma¬ 
trix,  and  perturbations  that  annihilate  the  off-diagonal  block  in  a  block  triangular  matrix. 
The  technique  can  be  used  to  give  very  simple  proofs  of  many  existing  relative  perturbation 
and  deflation  bounds,  and  some  new  relative  perturbation  and  deflation  results  for  the  sin¬ 
gular  values,  vectors,  and  invariant  subspaces  of  biacyclic,  triangular,  and  shifted  triangular 
matrices. 

The  final  focus  has  been  on  taking  advantage  of  structural  symmetry  in  sparse  elimina¬ 
tion.  We  have  shown  how  to  improve  the  performance  of  a  class  of  partial  pivoting  codes 
for  the  LU  factorization  of  large  sparse  unsymmetric  matrices.  Experimental  results  demon¬ 
strate  the  effectiveness  of  this  technique  in  reducing  the  overall  factorization  time.  Moreover, 
this  approach  has  been  generalized  to  create  a  new,  supernodal  algorithm  for  this  problem. 
The  new  algorithm  makes  heavy  use  of  the  BLAS-2  and  thus  runs  significantly  faster. 
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